Action recognition using instrumented objects for stroke rehabilitation

Assisting patients to perform activities of daily living (ADLs) is a challenging task for both human and machine. Hence, developing a computer-based rehabilitation system to re-train patients to carry out daily activities is an essential step towards facilitating rehabilitation of stroke patients with apraxia and action disorganization syndrome (AADS). This thesis presents a real-time Hidden Markov Model (HMM) based human activity recognizer, and proposes a technique to reduce the time delay occurred during the decoding stage. Results are reported for complete tea-making trials. In this study, the input features are recorded using sensors attached to the objects involved in the tea making task, plus hand coordinate data captured using Kinect sensor. A coaster of sensors, comprising an accelerometer and three force-sensitive resistors, are packaged in a unit which can be easily attached to the base of an object. A parallel asynchronous set of detectors, each responsible for the detection of one sub-goal in the tea-making task, are used to address challenges arising from overlaps between human actions. In this work HMMs are used to exploit temporal dependencies between actions and emission distributions are modelled by two generative and discriminative modelling techniques namely Gaussian Mixture Models (GMMs) and Deep Neural Networks (DNNs). Our experimental results show that HMM-DNN based systems outperform the GMM-HMM based systems by 18%. The proposed activity recognition system with the modified HMM topology provides a practical solution to the action recognition problem and reduces the time delay by 64% with no loss in accuracy